Refined quality feedback in streaming services

ABSTRACT

A method for quality feedback in a streaming service is shown, wherein at least one media stream is streamed to a client for playback, the method comprising determining a quality feedback value according to at least one quality metric from a pre-defined set of quality metrics, wherein the pre-defined set of quality metrics contains a quality metric related to at least one of the playback of the at least one media stream and a corruption of one out of at least two types of frames contained in the at least one media stream, and reporting the quality feedback value to a server. A system, a client, a server, a computer program, a computer program product and a protocol for quality feedback are also shown.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.10/841,228 filed May 7, 2004.

FIELD OF THE INVENTION

This invention relates to a method, a computer program, a computerprogram product, a system, a client, a server and a protocol for qualityfeedback in a streaming service, wherein at least one media stream isstreamed to a client for playback.

BACKGROUND OF THE INVENTION

Streaming, on the one hand, refers to the ability of an applicationsettled in a client to play back synchronized media streams like speech,audio and video streams in a continuous way while those streams arebeing transmitted to the client over a data network. On the other hand,streaming also refers to real-time low-delay applications such asconversational applications.

Applications that can be built on top of streaming services can beclassified into on-demand and live information delivery applications.Examples of the first category are music and news-on-demandapplications. Live delivery of radio and television programs areexamples of the second category. Real-time low delay application are,for example, multimedia (video) telephony or Voice over IP and any typeof conversational multimedia application.

Streaming over fixed Internet Protocol (IP) networks is already a majorapplication today. While the Internet Engineering Task Force (IETF) andthe World Wide Web Consortium (W3C) have developed a set of protocolsused in fixed-EP streaming services, no complete standardized streamingframework has yet been defined. For Third Generation (3G) mobilecommunications systems according to the standards developed by the ThirdGeneration Partnership Project (3GPP), the 3G Packet-switched StreamingService (PSS, 3GPP TS 26.233, TS 26.234) fills the gap between the 3GMulti-media Messaging Service (MMS), for instance downloadingapplications and multimedia content, and conversational & streamingservices.

The PSS enables mobile streaming applications, wherein the complexity ofthe terminals is lower than that required for conversational services,because no media input devices and encoders are required, and becauseless complex protocols can be used. The PSS includes a basic set ofstreaming control protocols, transport protocols, media codes and scenedescription protocols.

FIG. 1 schematically depicts the PSS protocol stack 1 that controls thetransfer of both streamable and non-streamable content between a contentor media server and a client.

Streamable content 101, such as video, audio and speech, is firstconverted to the payload format of the Real-time Transport Protocol(RTP) 102 in an adaptation layer 103. Said RTP as defined by the IETFprovides means for sending real-time or streaming data by using theservices of an underlying User Datagram Protocol (UDP) 104, which inturn uses the services of an underlying IP protocol 105.

Non-streamable content 106, as for instance multimedia content which isnot created for streaming purposes (e.g. MMS clips recorded on aterminal device), still images, bitmap and vector graphics, text, timedtext and synthetic audio are transferred by the Hypertext TransferProtocol (HTTP) 107, which uses the services of the underlying TransportControl Protocol (TCP) 108 and the further underlying IP 105.

Whereas for the non-streamable content 106, the built-in session set-upand control capabilities of the HTTP 107 are sufficient to transfer thecontent, in case of streamable content 101, an advanced session set-upand control protocol has to be invoked, for instance to start, stop andpause a streaming video that is transferred from the content server tothe client via the RTP/UDP/IP. This task is performed by the Real-timeStreaming Protocol (RTSP) 109, which may either use the underlying TCP108 or the underlying UDP 104. RTSP requires a presentation description110 at least to set-up a streaming session. Such a presentationdescription 110 may for instance be available in the form of a SessionDescription Protocol (SDP) file. Said SDP file contains the descriptionof the session, for instance session name and author, the type of mediato be presented, information to receive said media, as for instanceaddresses, ports, formats and so on, and the bitrate of the media.

If streaming content is to be viewed at the client side, for instance ata mobile terminal, the user of said terminal is first provided with aUniversal Resource Identifier (URI) to specific content that suits histerminal. This URI may come form a WWW server, a Wireless ApplicationProtocol (WAP) server, or may have been entered manually via thekeyboard of the terminal. This URI specifies a streaming or RTSP serverand the address of the content on that or another content server. Thecorresponding SDP file may now be obtained in a number of ways. It maybe provided in a link inside the HTML page that the user downloads, forinstance via an embed tag, or may also be directly obtained by typing itas a URI. The SDP file, i.e. the presentation description 110, then istransferred via the HTTP 107 as indicated in the middle column of theprotocol stack of FIG. 1. Alternatively, it may also be obtained throughRTSP 109 signaling, for instance by using the DESCRIBE method of theRTSP 109, as indicated by the right column of the protocol stack inFIG. 1. Note that the presentation description may equally well betransmitted by said RTP 102. However, for simplicity of presentation,this possibility was not included in FIG. 1.

The subsequent session establishment is the process in which the browseror the user of the mobile terminal invokes a streaming client to set upthe session against the content server. The terminal is expected to havean active radio bearer that enables IP-based packet transmission at thestart of session establishment signaling.

The subsequent set-up of the streaming service is done by sending anRTSP SETUP message for each media stream chosen by the client. Thisreturns the UDP 104 and/or TCP 108 port to be used for the respectivemedia stream. The client sends an RTSP PLAY message to the contentserver that then starts to send one or more streams over the IP network.

In order to offer service providers in PSS systems means to evaluate theend user streaming experience, streaming service quality metrics havebeen introduced in PSS systems, as presented in 3GPP Technical document(Tdoc) S4-040073: “Draft Rel-6 PSS Quality Metrics Permanent Documentv.0.11”, which refers to 3GPP TSG-SA4 meeting #30 in Malaga, Spain, Feb.23-27, 2004. The streaming client measures and feeds back information onthe quality of the actual streaming application (Quality of Experience,QoE) to a streaming server, wherein said quality is defined in terms ofsaid quality metrics. Said streaming server may for instance be an RTSPserver, and said quality metrics may for instance be transported byusing said RTSP and SDP.

Because the service is transparent to the type of RAN and CN, only thestreaming client and the streaming server are impacted by the PSSquality metrics. One consequence of this is that the measurements maynot rely on information from protocol layers below the RTP layer (e.g.UDP, IP, PDCP, SNDCP, LLC, RLC, MAC, Physical Layer).

The terminal in a PSS system with quality feedback is responsible toperform the quality measurements in accordance to the measurementdefinition, aggregate them into streaming client quality metrics andreport the metrics to the streaming server. This requirement does notpreclude the possibility for the streaming client to report raw qualitymeasurements to be processed by the streaming server into qualitymetrics.

The streaming server is responsible to signal the activation of thestreaming client's quality metrics reporting and to gather the streamingclient's quality metrics. The streaming server may process the receivedstreaming client's quality metrics to build aggregated quality metrics.E.g. it could receive a raw lost packets report and build the Min, Max,Avg and Std packet loss rate for a particular streaming client.

The following four quality metrics are defined by Tdoc S4-040073:

Corruption Duration

Corruption duration is the time period from the first corrupted frame tothe first subsequent good frame or the end of the reporting period(whichever is sooner). The unit of this metrics is expressed in seconds,and can be a fractional value.

Rebuffering Duration

This metric is only applicable for audio, video and speech, and is notapplicable to other media types. The unit of this metrics is expressedin seconds, and can be a fractional value. Rebuffering is defined as anystall in playback time due to any involuntary event at the client side.

Initial Buffering Time

Initial buffering is the time from receiving the first RTP packet untilplayback starts. The unit of this metrics is expressed in seconds, andcan be a fractional value.

Number of Content Packets Lost in Succession

The number of content packets lost in succession per media channel.

The objective of the above quality metric definition is to obtainconsistent measurements across content type, terminals, and types ofRadio Access Network (RAN).

The constraints are to minimize the size of the quality metrics reportthat will be sent to the streaming server and, the complexity for theterminal.

The actual quality metrics feedback can be conveyed to the PSS server byusing the SET_PARAMETER method of the RTSP with a feedback header 2 asdepicted in FIG. 2 (with reference to IETF Request for Comments (RFC)document 2327), however, in particular cases, it is more efficient touse other methods to carry the information, as for instance the TEARDOWNmessage or the PAUSE message.

In the feedback header 2 of FIG. 2, Stream-url is the RTSP session ormedia control URL identifier for the feedback parameter. The Metricsfield in the Parameters definition contains the name of themetrics/measurements (for instance corruption duration, etc.). The Valuefield indicates the results. There is the possibility that the sameevent occurs more than once during a monitoring period. In that case themetrics value can occur more than once, which indicates the number ofevents to the server. The optional Range field indicates the reportingperiod.

The optional Timestamp field in the feedback header 2 of FIG. 2indicates the time when the event (or measurement) occurred or when themetric was calculated since the beginning of the session.

The four quality metrics defined by Tdoc S4-040073 only allow for acoarse characterization of the quality of the playback of multimediastreams as experienced by a user. For instance, if two streamingsessions have the same values of the four quality metrics defined byTdoc S4-040073, and if in the first of said sessions, a perfectsynchronization between audio and video data exists, whereas in thesecond of said sessions, said synchronization between audio and videohas been lost, the reported quality based on the four quality metricsdefined by Tdoc S4-040073 is the same while the actually experiencedquality of playback is quite different. Furthermore, the four qualitymetrics defined by Tdoc S4-040073 do not differentiate between thedifferent frame types contained in said multimedia stream, so that, forinstance, the loss of frame types that are of crucial importance for theexperienced quality of the playback can not be differentiated from theloss of less important types of frames when reporting quality.

SUMMARY OF THE INVENTION

In view of the above-mentioned problems, it is, inter alia, an object ofthe present invention to provide a method, a computer program, acomputer program product, a system, a client, a server and a protocolthat allow for a refined quality feedback in a streaming service.

A method for quality feedback in a streaming service is proposed,wherein at least one media stream is streamed to a client for playback,said method comprising determining a quality feedback value according toat least one quality metric from a pre-defined set of quality metrics,wherein said pre-defined set of quality metrics contains a qualitymetric related to at least one of said playback of said at least onemedia stream and a corruption of one out of at least two types of framescontained in said at least one media stream, and reporting said qualityfeedback value to a server.

Said at least one media stream may for instance be a continuous mediastream that may contain video, audio or speech information that iscontinuously transmitted from a server, for instance a content server,to said client and is played back on the terminal, in which said clientis set up, in a synchronized manner. Alternatively, said at least onemedia stream may be a media stream of a real-time low delay application,as for example a multimedia (video) telephony stream or a Voice-over-IPmedia stream or any type of media stream in a conversational multimediaapplication. This streaming may take place in a streaming session,wherein several media streams may be concurrently streamed to saidclient. Said streaming may be based on a protocol, for instance theReal-time Transport Protocol RTP, and may be controlled by a furtherprotocol, for instance a streaming protocol as the Real-time StreamingProtocol RTSP or the Session Initiation Protocol SIP, and may forinstance allow to start, stop and/or pause the streaming. Said RTSP orSIP may be operated by protocol entities in said client and in saidserver and may be based on a Session Description Protocol SDP. Saidserver may be co-located or even be identical with the content serverfrom which said media actually stems from, or may be a differentinstance.

The quality of said streaming is determined at the client site accordingto said at least one quality metric from said pre-defined set of qualitymetrics, and reported as a quality feedback value, for instance via saidprotocol that the streaming is based on or said protocol that controlsthe streaming. Said quality metric basically defines how said qualityfeedback value is calculated. Said pre-defined set of quality metricsmay be defined by said protocol that controls the streaming, and said atleast one quality metric, may be negotiated between said client and saidserver before, during or even after the set-up of the session.

Said quality feedback value is reported to said server, for instance viasaid protocol the streaming is based on or via said protocol thatcontrols the streaming. If said protocol that controls said streaming isan RTCP or SIP, it may be preferred that said reported quality feedbackvalues are captured or sniffed by an entity, for instance a networkentity such as a Call State Control Function CSCF, in order to makequality measurements.

According to the present invention, to refine quality feedback, saidpre-defined set of quality metrics contains a quality metric related tosaid playback of said at least one media stream, and/or a quality metricrelated to a corruption of one out of at least two types of framescontained in said at least one media stream. Said quality metric relatedto said playback of said at least one media stream may for instancerelate to a deviation of the actual playback frame rate of said playbackfrom a pre-defined value, or to a difference between the actual playbacktime and a scheduled playback time, or to a loss of synchronizationbetween the playback of audio or speech frames and the correspondingvideo frames. Said quality metric related to a corruption of one out ofat least two types of frames contained in said at least one media streammay for instance relate to a corruption of scene cut frames of a videostream that may be essential for the experienced quality of theplayback.

The incorporation of quality metrics related to playback and/or thecorruption of specific types of frames according to the presentinvention thus allows for a refinement of quality feedback.

According to a preferred embodiment of the present invention, saidquality metric related to said playback of said at least one mediastream is a frame-rate-deviation metric that is related to a deviationof a playback frame rate of said playback of said at least one mediastream from a pre-defined value. When playing back said at least onemedia stream, which may be composed of a sequence of frames, a framerate with which said frames are played back may not equal a frame ratethat is desired or optimum for said playback. It is thus instructive toreport such as deviation. Said pre-defined value may be a default valueknown by both the server and the client, or only by one of the two. Saidpre-defined value may for instance be an optimum playback frame rate.Said frame-rate-deviation metric may further be related to a duration ofsaid deviation event.

According to a preferred embodiment of the present invention, saidquality metric related to said playback of said at least one mediastream is a jitter-duration metric that is related to a duration of ajitter event that occurs when a difference between a playback time ofsaid playback of said at least one media stream and a scheduled playbacktime is larger than a pre-defined value. Said pre-defined value may be adefault value known by both the server and the client, or only by one ofthe two. Said playback time denotes the actual playback time, which mayfor instance be measured in Normal Play Time (NPT) or any other timesystem at the client, e.g. wall-clock time. Different time systems mayhave different origins (with the value 0) and units. Each media frame isassociated with a scheduled playback time, which may for instancebe-signaled using Real-Time Transmission Protocol (RTP) timestamps.However, due to reasons such as network delay, a media frame might notbe played back exactly at the scheduled playback time, so that playbacktime and scheduled playback time differ.

According to a preferred embodiment of the present invention, saidquality metric related to said playback of said at least one mediastream is a synchronization-loss-duration metric that is related to aduration of a synchronization loss event that occurs when the absolutedifference between a value A and a value B is larger than a pre-definedvalue, wherein said value A is a difference between a playback time of alast played frame of a first media stream of said at least one mediastreams and a playback time of a last played frame of a second mediastream of said at least one media streams, and wherein said value B is adifference between a scheduled playback time of said last played frameof said first media stream of said at least one media streams and ascheduled playback time of said last played frame of said second mediastream of said at least one media streams. Said first media stream mayfor instance be an audio or speech stream, and said second media streammay for instance be a video stream. If playback refers to thesynchronous playback of video and speech/audio streams, a lack ofsynchronization of images and corresponding speech/audio may beexperienced as annoying by a user and thus is important to be reportedto said server. Said pre-defined value may be a default value known byboth the server and the client, or only by one of the two.

According to a preferred embodiment of the present invention, one out ofsaid at least two types of frames contained in said at least one mediastream is a scene cut frame, and said quality metric related to acorruption of one out of at least two types of frames contained in saidat least one media stream is a number-of-corrupted-scene-cut-framesmetric that is related to a number of corrupted scene cut frames in saidat least one media stream. Said other type of frames then may forinstance be a normal content frame that is similar as the previous framein content. It is thus possible to separately report on the corruptionof scene cut frames that may be of particular importance for theplayback quality. Said scene cut frame may for instance indicate a cutof a scene in a video stream, information related to said scene cutframes may be available at the server or may be derived at the server orthe client, and said information may be signaled in-band or out-of-bandto the client.

According to a preferred embodiment of the present invention, saidquality metric is reported together with a timestamp. Said timestamp mayfor instance indicate an absolute or relative time instance of anoccurrence of an event that is described according to said qualitymetric.

According to a preferred embodiment of the present invention, saidframe-rate-deviation metric is reported together with a timestamp, andsaid timestamp is equal to a play time of a first played frame duringsaid deviation relative to a starting time of a quality reportingperiod. Said play time may for instance be represented as NPT.

According to a preferred embodiment of the present invention, saidjitter-duration metric is reported together with a timestamp, and saidtimestamp is equal to a play time of a first played frame during saidjitter event relative to a starting time of a quality reporting period.Said play time may for instance be represented as NPT.

According to a preferred embodiment of the present invention, saidsynchronization-loss-duration metric is reported together with atimestamp, and said timestamp is equal to a play time of a first playedframe during said synchronization loss event relative to a starting timeof a quality reporting period. Said play time may for instance berepresented as NPT.

According to a preferred embodiment of the present invention, said playtime is a Normal Play Time NPT. Said NPT may for instance be provided bya Real-Time Streaming Protocol RTSP. Defining said play time via the NPTremoves ambiguities in the interpretation of timestamps by clients andservers and thus allows for a correct analysis of streaming qualityexperiences.

According to a preferred embodiment of the present invention, saidpre-defined set of quality metrics contains a corruption-duration metricthat is related to a duration of a corruption, and said quality feedbackvalue determined according to said corruption-duration metric is onlyreported if said duration is larger than a pre-defined value. Duringplayback, if only a single frame or a few frames in a short time periodare not played back, the end user typically cannot perceive thedifference. The receiving terminal may decide not to play back a framefor instance due to one of the following reasons: Due to any reason, anon-reference frame has not been transmitted from the server; anon-reference frame is partially or entirely lost; a non-reference frameis completely received, but it cannot be correctly decoded; anon-reference frame is completely received, but due to delay or lackingof computing capability, it is not decoded; a reference or non-referenceframe is completely received and correctly decoded but its scheduleddisplay time has expired. Non-playback of such frames in a short timeperiod does not affect user experience, so that reporting of such framesof a short time period would be a waste of transmission bandwidth. Thus,according to this preferred embodiment of the present invention, it isproposed that a corruption duration is only reported when it exceedssaid pre-defined value, which may for instance be a default value set bythe server and/or the client or a value prescribed by a protocol.

According to a preferred embodiment of the present invention, said atleast one media stream is based on a Real-time Transport Protocol RTP.Said RTP may be operated between said client and a content server andmay use the services of a User Datagram Protocol UDP, which in turn mayuse the services of an Internet Protocol IP.

According to a preferred embodiment of the present invention, saidstreaming is at least partially controlled by a Real-time StreamingProtocol RTSP. Said RTSP may be based on a presentation descriptionprovided by a Session Description Protocol SDP. Said RTSP may beoperated by said client and said server and may for instance allow forthe starting, pausing and stopping of the streaming.

According to a preferred embodiment of the present invention, saidquality feedback value is reported to said server via said RTSP. Saidquality feedback value may for instance be contained in a header of anRTSP protocol data unit.

According to a preferred embodiment of the present invention, saidstreaming service is a Packet-switched Streaming Service PSS in a 3Gmobile communications system.

A computer program is further proposed with instructions operable tocause a processor to perform the above-mentioned method steps. Saidcomputer program may for instance be run on a processor in said client.

A computer program product is further proposed comprising a computerprogram with instructions operable to cause a processor to perform theabove-mentioned method steps. Said computer program product may forinstance be stored in a computer readable medium or memory, such as aROM or RAM, or be stored on a removable medium such as a memory card, amemory stick, a disk, a CD or similar media, and may be loaded into theinternal memory of a processor that is located in said client.

A system for quality feedback in a streaming service is furtherproposed, comprising at least one server, and at least one client,wherein at least one media stream is streamed to said at least oneclient for playback, wherein a quality feedback value is determinedaccording to at least one quality metric from a pre-defined set ofquality metrics, wherein said pre-defined set of quality metricscontains a quality metric related to at least one of said playback ofsaid at least one media stream, and a corruption of one out of at leasttwo types of frames contained in said at least one media stream, andwherein said quality feedback value is reported to said at least oneserver.

A client in a streaming service is further proposed, comprising meansfor receiving at least one media stream that is streamed to said clientfor playback, means for determining a quality feedback value accordingto at least one quality metric from a pre-defined set of qualitymetrics, wherein said pre-defined set of quality metrics contains aquality metric related to at least one of said playback of said at leastone media stream and a corruption of one out of at least two types offrames contained in said at least one media stream, and means forreporting said quality feedback value to a server.

A server in a streaming service is further proposed, wherein at leastone media stream is streamed to a client for playback, wherein a qualityfeedback value is determined according to at least one quality metricfrom a pre-defined set of quality metrics, wherein said pre-defined setof quality metrics contains a quality metric related to at least one ofsaid playback of said at least one media stream and a corruption of oneout of at least two types of frames contained in said at least one mediastream, said server comprising means for receiving said quality feedbackvalue that is reported by said client to said server.

A protocol to be used in a streaming service is further proposed,wherein at least one media stream is streamed to a client for playback,the protocol defining a quality metric related to at least one of saidplayback of said at least one media stream, and a corruption of one outof at least two types of frames contained in said at least one mediastream. Said protocol may be an RTSP in combination with a SessionDescription Protocol SDP.

These and other aspects of the invention will be apparent from andelucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE FIGURES

In the figures show:

FIG. 1: A schematic representation of a Packet-Switched StreamingService (PSS) protocol stack according to the prior art,

FIG. 2: a definition of a Real-time Streaming Protocol (RTSP)negotiation header according to the prior art,

FIG. 3: a flowchart of the method of the present invention, and

FIG. 4: a schematic representation of a system according to the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention proposes special quality metrics to be used inquality feedback for streaming services in order to refine qualityfeedback. In the following, the proposed quality metrics and theirassociated timestamps will be described in more detail.

Frame-Rate-Deviation

This quality metric may only be applicable for audio, video and speech,and it may not be applicable to other media types. It gives informationon the playback frame rate. Frame rate deviation happens when theplayback frame rate deviates from a pre-defined value. This qualitymetric may contain both the time duration of the event and the framerate deviation value, i.e. the difference between a pre-defined framerate and the actual playback frame rate. The time duration may beexpressed in units of seconds, and may be a fractional value. Thedeviation value may be expressed in units of frames per second, and mayalso be a fractional value.

Said pre-defined value may be a default value known by both the serverand the client, or it may be provided by the server during QoEnegotiation. If it is to be provided by the server, the server maydecide the value by checking the media bitstream or any other means.

It may be advantageous that the value indicates the average frame ratecalculated when the media stream is locally played back.

From an implementation point of view, the following method may bespecified to calculate the frame rate. It is assumed that the frame ratechanges only at integer seconds. The frame rate value of the time periodfrom second n-1 to second n is equal to the number of frames played backduring the period. The time axis may represent NPT time and mayoriginate from the starting time of the QoE reporting period.

The timestamp associated with the frame-rate-deviation metric indicatesthe time when the frame rate deviation has occurred. The value of thetimestamp may be equal to the NPT of the first played frame during theframe rate deviation event, relative to the starting time of the QoEreporting period. If there is no played frame during the event, thevalue may be equal to the NPT of the last played frame before the eventor the starting time of the QoE reporting period, whichever is later,relative to the starting time of the QoE reporting period.

Jitter-Duration

This quality metric may be only applicable for audio, video and speech,and may not be applicable to other media types. A playback jitterhappens when the absolute difference between the actual playback timeand the scheduled playback time is larger than a pre-defined value. Thismetric may be expressed in units of seconds, and may be a fractionalvalue. Said pre-defined value may be a default value known by both theserver and the client, e.g. 100 milliseconds.

The timestamp associated with the jitter-duration metric indicates thetime when the playback jitter has occurred. The value of the timestampmay be equal to the NPT of the first played frame in the playbackjitter, relative to the starting time of the QoE reporting period.

Synchronization-Loss-Duration

This quality metric may be applicable for any pair of media types.Define a value A as a difference between a playback time of a lastplayed frame of a first media stream of said at least one media streamsand a playback time of a last played frame of a second media stream ofsaid at least one media streams, and define a value B as a differencebetween a scheduled playback time of said last played frame of saidfirst media stream of said at least one media streams and a scheduledplayback time of said last played frame of said second media stream ofsaid at least one media stream. A synchronization loss happens when theabsolute difference between said value A and said value B is larger thana pre-defined value. This quality metric may be expressed in units ofseconds, and may be a fractional value. Said pre-defined value may be adefault value known by both the server and the client, e.g. 100milliseconds.

The timestamp associated with the synchronization-loss-duration metricindicates the time when the playback synchronization loss has occurred.The value of the timestamp may be equal to the NPT of the first playedframe in the synchronization loss, relative to the starting time of theQoE reporting period.

Number-of-Corrupted-Scene-Cut-Frames

This quality metric may be only applicable for video, and may not beapplicable to other media types. This metric refers to the number ofcorrupted scene cut frames during the QoE reporting period. This metricmay be expressed in units of integers larger than or equal to zero. Ifthis metric is supported, and if no such metric is reported for a QoEreporting period, this may indicate that no scene cut frame has beencorrupted during the QoE reporting period.

This quality metric may be used if the server has information indicatingwhich frames are scene cut frames, or if the server implements a methodto derive the information, for instance by using a scene cut detectionalgorithm. In this case, it may be advantageous that the server makesthe information available to the client either via in-band orout-of-band signaling. This quality metric may also be used if theclient implements a method to derive the information, for instance byusing a scene cut detection algorithm. For example, in ITU-T H.264(a.k.a. ISO/IEC MPEG-4 Part 10), scene cut information can be conveyedto the client in-band using the scene information supplementalenhancement information (SEI) message.

The timestamp associated with the number-of-corrupted-scene-cut-fra-mesmetric may not be defined, since it may be clear that the time when thenumber of corrupted scene cut frames is measured is equal to the endingtime of the QoE reporting period.

An example protocol syntax for each of the new quality metrics asproposed by the present invention for instance reads as:Framerate_Deviation=“Framerate_Deviation”“=”“{”SP/(Value1SP Value2[SPTimestamp])#(“,”Value1SP Value2[SP Timestamp])“}”;Jitter_Duration=“Jitter_Duration”“=”“{”SP/(Value[SPTimestamp])#(“,”Value[SP Timestamp])“}”;Syncloss_Duration=“Syncloss_Duration”“=”“{”SP/(Value[SPTimestamp])#(“,”Value[SPTimestamp])“}”;Corrupted_Scene_Cuts=“Corrupted_Scene_Cuts”“=”“{”SP/Value“}”;

Therein, the Value1 field indicates the time duration of the frame ratedeviation event, and the Value2 field indicates the frame rate deviationvalue. The Value field indicates the time duration of the playbackjitter, the time duration of synchronization loss, or the number ofcorrupted scene cut frames. The semantics of the Timestamp field are asspecified above.

In quality feedback, there exists the possibility that the same eventoccurs more than once during a monitoring period. In that case, aquality feedback value may occur more than once, which indicates thenumber of events to the server.

To enable the client to calculate the frame-rate-deviation metric, apre-defined frame rate value may need to be sent to the client. In thiscase, the value can be added as one additional parameter to an RSTPheader QoE-Header or SDP attribute “QoE-Metrics”, as specified in TdocS4-040073. The syntax design for the pre-defined frame rate value may,for example, be defined as follows:Pre-defined Frame Rate=“FR”“=”1*DIGIT“.”1*DIGIT

The syntaxes of the RTSP QoE-Header and the SDP QoE-Metrics then maybecome:QoE-Header=“QoE-Metrics”“:”“off”/1#(stream-url“;”Metrics“;”Sending-rate[“;”Range][“;”FR])CRLFanda=QoE-Metrics: Metrics“;”Sending-rate[“;”Range][“;”FR]CRLF

To use transmission bandwidth more efficiently when performing qualityfeedback in streaming services, the present invention proposes that aquality feedback value determined according to the corruption-durationmetric is only reported if said duration is larger than a pre-definedvalue. During playback, if only a single frame or a few frames in ashort time period are not played back, the end user typically cannotperceive the difference. The receiving terminal may decide not to playback a frame for instance due to one of the following reasons: Due toany reason, a non-reference frame has not been transmitted from theserver; a non-reference frame is partially or entirely lost; anon-reference frame is completely received, but cannot be correctlydecoded; a non-reference frame is completely received, but due to delayor lacking of computing capability, it is not decoded; a reference ornon-reference frame is completely received and correctly decoded but itsscheduled display time has expired. Non-playback of such frames in ashort time period does not affect user experience, so that reporting ofsuch frames of a short time period would be a waste of transmissionbandwidth. Thus, according to the present invention, it is proposed thata corruption duration is only reported when it exceeds said pre-definedvalue, which may for instance be a default value set by the serverand/or the client or a value prescribed by a protocol.

FIG. 3 depicts a flowchart of a method according to the presentinvention. In a first step 300, a streaming session is set up between astreaming client and a streaming server. In a step 301, one or morequality metrics from a pre-defined set of quality metrics, whichcomprises the quality metrics as proposed by the present invention, arenegotiated between the streaming client and the streaming server for usein the quality feedback procedure that is performed by the streamingclient. Both said session set-up and negotiation may be based on an RTSPin combination with an SDP, or on an RTCP or SIP. Step 301 may also beperformed together with step 300. A corresponding timestamp metric maybe associated with at least some of the negotiated quality metrics forthe streaming session. In a step 302, the actual streaming is started,for instance when a media stream is transmitted to the streaming clientand played back on the terminal in which said streaming client is setup. During said streaming, in a step 303, it is checked if a qualityfeedback is required or not. This may for instance be accomplished bycontinuously checking if an event, which has to be reported to thestreaming server according to the negotiated quality metric, occurs ornot. This may for instance be a loss of synchronization event.Alternatively, a periodical quality report may have been negotiated, forinstance the periodical feed-back of the number of corrupted scene cutframes in a certain time interval. In said step 303, both theevent-driven and the periodical quality feed-back is triggered. If it isdecided that quality feedback is required, in a step 304, a qualityfeedback value is determined according to each negotiated qualitymetric. If one or several of said negotiated quality metrics areassociated with a timestamp metric, corresponding timestamps aredetermined in a step 305. Said step 305 may equally well be performedbefore the step 304. The quality feedback values and the correspondingtimestamps are then reported to the streaming server in a step 306, forinstance via the RTSP, RTCP or SIP. After quality feedback, or if it isdecided that no quality feedback is required, it is checked in a step307 if streaming is to be stopped. If this is not the case, it is againchecked in a step 303 if a new quality feedback is required or not.

FIG. 4 schematically depicts the functional components of a systemaccording to the present invention. This embodiment exemplarily refersto a PSS system that uses an RTSP to control the streaming. It isunderstood that equally well, the SIP could be used here with a slightlymodified underlying protocol stack and with an additional networkinstance that sniffs or captures the quality feedbacks and timestampsthat are sent from the client 601 (a first party) to the server 600 (asecond party). The PSS system in FIG. 4 comprises a streaming client 601and a streaming server 600, wherein both client 601 and server 600 haveat least one RTSP entity 401, 400 that is capable of operating the RTSP.The RTSP entities 400, 401 use the services of underlying protocollayers that are operated by further protocol entities, of which only theTCP/UDP entities 402, 403 and the IP entities 404, 405 are shown. Thestreaming client 601 is further connected to a streaming quality monitorinstance 407, which monitors the quality of the actual streamingapplication in terms of the negotiated quality metrics and possibly acorresponding timestamp metric and inputs monitored quality feedbackvalues into said RTSP entity 401. Said streaming quality monitor may forinstance be provided by the terminal, in which said streaming client isset up. If a timestamp metric is defined for the used quality metric,the streaming quality monitor 407 then determines a timestamp accordingto said timestamp metric, and transfers said monitored quality feedbackvalues and said corresponding timestamps, via the client RTSP 401, tothe RTSP peer entity in the streaming server 600, where they are inputinto a quality data processing instance 406 for evaluation and analysis,which may for instance aim at improving the quality of the streamingapplication by enhancing the error resilience of the streams if it isfound that the corruption duration events become too frequent, or justaim at statistical quality data collection or charging or other aims.

The invention has been described above by means of a preferredembodiment. It should be noted that there are alternative ways andvariations which are obvious to a skilled person in the art and can beimplemented without deviating from the scope and spirit of the appendedclaims. In particular, the present invention is by no means restrictedto application in 3G radio communications systems. It may equally wellbe deployed in all kinds of wired and wireless data transmission systemswith parameter feedback.

1. A method comprising: determining a difference between a playback timeof a playback of at least one media stream streamed to a client forplayback and a scheduled playback time; comparing said differencebetween said playback time of said playback of said at least one mediastream and said scheduled playback time to a pre-defined value;detecting a jitter event that is considered to occur when said comparingsaid difference between said playback time of said playback of said atleast one media stream and said scheduled playback time to saidpre-defined value yields that said difference is larger than saidpre-defined value; determining a quality feedback value according to atleast one quality metric related to said playback of said at least onemedia stream, said quality metric being a jitter-duration metric that isrelated to a duration of said detected jitter event; and reporting saidquality feedback value to a server together with a timestamp that isequal to a play time of a first played frame during said jitter eventrelative to a starting time of a quality reporting period.
 2. The methodaccording to claim 1, wherein said jitter-duration metric is containedin a pre-defined set of quality metrics that further contains aframe-rate-deviation metric that is related to a deviation of a playbackframe rate of said playback of said at least one media stream from apre-defined value.
 3. The method according to claim 2, wherein saidframe-rate-deviation metric further is related to a duration of saiddeviation.
 4. The method according to claim 1, wherein saidjitter-duration metric is contained in a pre-defined set of qualitymetrics that further contains a synchronization-loss-duration metricthat is related to a duration of a synchronization loss event thatoccurs when the absolute difference between a value A and a value B islarger than a pre-defined value, wherein said value A is a differencebetween a playback time of a last played frame of a first media streamof said at least one media streams and a playback time of a last playedframe of a second media stream of said at least one media streams, andwherein said value B is a difference between a scheduled playback timeof said last played frame of said first media stream of said at leastone media streams and a scheduled playback time of said last playedframe of said second media stream of said at least one media streams. 5.The method according to claim 1, wherein said jitter-duration metric iscontained in a pre-defined set of quality metrics that further containsa quality metric related to a corruption of one out of at least twotypes of frames contained in said at least one media stream, wherein oneout of said at least two types of frames contained in said at least onemedia stream is a scene cut frame, and wherein said quality metricrelated to a corruption of one out of at least two types of framescontained in said at least one media stream is anumber-of-corrupted-scene-cut-frames metric that is related to a numberof corrupted scene cut frames in said at least one media stream.
 6. Themethod according to claim 1, wherein said quality metric is reportedtogether with a timestamp.
 7. The method according to claim 2, whereinsaid frame-rate-deviation metric is reported together with a timestamp,and wherein said timestamp is equal to a play time of a first playedframe during said deviation relative to a starting time of a qualityreporting period.
 8. The method according to claim 4, wherein saidsynchronization-loss-duration metric is reported together with atimestamp, and wherein said timestamp is equal to a play time of a firstplayed frame during said synchronization loss event relative to astarting time of a quality reporting period.
 9. The method according toclaim 7, wherein said play time is a Normal Play Time NPT.
 10. Themethod according to claim 1, wherein said jitter-duration metric iscontained in a pre-defined set of quality metrics that further containsa corruption-duration metric that is related to a duration of acorruption, and wherein said quality feedback value determined accordingto said corruption-duration metric is only reported if said duration islarger than a pre-defined value.
 11. The meted according to claim 1,wherein said streaming of said at least one media stream is based on aReal-time Transport Protocol RTP.
 12. The method according to claim 1,wherein said streaming is at least partially controlled by a Real-timeStreaming Protocol RTSP.
 13. The method according to claim 12, whereinsaid quality feedback value is reported to said server via said RTSP.14. The method according claim 1, wherein said streaming is performedaccording to the Packet-switched Streaming Service PSS in a thirdgeneration mobile communications system.
 15. A computer-readable mediumhaving a program product stored therein, the computer program comprisinginstructions operable to cause a processor to perform the method ofclaim
 1. 16. An apparatus comprising: means for determining a differencebetween a playback time of a playback of at least one media streamstreamed to said apparatus for playback and a scheduled playback time;means for comparing said difference between said playback time of saidplayback of said at least one media stream and said scheduled playbacktime to a pre-defined value; means for detecting a jitter event that isconsidered to occur when said comparing said difference between saidplayback time of said playback of said at least one media stream andsaid scheduled playback time to said pre-defined value yields that saiddifference is larger than said pre-defined value; means for determininga quality feedback value according to at least one quality metricrelated to said playback of said at least one media stream, said qualitymetric being a jitter-duration metric that is related to a duration ofsaid detected jitter event; and means for reporting said qualityfeedback value to a server together with a timestamp that is equal to aplay time of a first played frame during said jitter event relative to astarting time of a quality reporting period.
 17. An apparatuscomprising: means for receiving a quality feedback value and a timestampreported together by a client to said apparatus, the quality feedbackvalue having been determined by determining a difference between aplayback time of a playback of at least one media stream streamed tosaid client for playback and a scheduled playback time; comparing saiddifference between said playback time of said playback of said at leastone media stream and said scheduled playback time to a pre-definedvalue; detecting a jitter event that is considered to occur when saidcomparing said difference between said playback time of said playback ofsaid at least one media stream and said scheduled playback time to saidpre-defined value yields that said difference is larger than saidpre-defined value; and determining said quality feedback value accordingto at least one quality metric related to said playback of said at leastone media stream, said quality metric being a jitter-duration metricthat is related to a duration of said detected jitter event, whereinsaid timestamp is equal to a play time of a first played frame duringsaid jitter event relative to a starting time of a quality reportingperiod.
 18. The method according to claim 1, wherein said play time is aNormal Play Time NPT.
 19. The method according to claim 8, wherein saidplay time is a Normal Play Time NPT.
 20. An apparatus comprising: amonitor configured to determine a difference between a playback time ofa playback of at least one media stream streamed to said apparatus forplayback and a scheduled playback time; compare said difference betweensaid playback time of said playback of said at least one media streamand said scheduled playback time to a pre-defined value; detect a jitterevent that is considered to occur when said comparing said differencebetween said playback time of said playback of said at least one mediastream and said scheduled playback time to said pre-defined value yieldsthat said difference is larger than said pre-defined value; determine aquality feedback value according to at least one quality metric relatedto said playback of said at least one media stream, said quality metricbeing a jitter-duration metric that is related to a duration of saiddetected jitter event; and a transmitter configured to report saidquality feedback value to a server together with a timestamp that isequal to a play time of a first played frame during said jitter eventrelative to a starting time of a quality reporting period.
 21. Theapparatus according to claim 20, wherein said quality metric is reportedtogether with a timestamp.
 22. The apparatus according to claim 20,wherein said play time is a Normal Play Time NPT.
 23. The apparatusaccording to claim 20, wherein said streaming of said at least one mediastream is based on a Real-time Transport Protocol RTP.
 24. The apparatusaccording to claim 20, wherein said streaming is at least partiallycontrolled by a Real-time Streaming Protocol RTSP.
 25. The apparatusaccording to claim 24, wherein said quality feedback value is reportedto said server via said RTSP.
 26. The apparatus according claim 20,wherein said apparatus is a client of the Packet-switched StreamingService PSS of a third generation mobile communications system.
 27. Anapparatus comprising: a receiver configured to receive a qualityfeedback value and a timestamp reported together by a client to saidapparatus, the quality feedback value having been determined bydetermining a difference between a playback time of a playback of atleast one media stream streamed to said client for playback and ascheduled playback time; comparing said difference between said playbacktime of said playback of said at least one media stream and saidscheduled playback time to a pre-defined value; detecting a jitter eventthat is considered to occur when said comparing said difference betweensaid playback time of said playback of said at least one media streamand said scheduled playback time to said pre-defined value fields thatsaid difference is larger than said pre-defined value; determining saidquality feedback value according to at least one quality metric relatedto said playback of said at least one media stream, said quality metricbeing a jitter-duration metric that is related to a duration of saiddetected jitter-event, wherein said timestamp is equal to a play time ofa first played frame during said jitter event relative to a startingtime of a quality reporting period.
 28. The apparatus according to claim27, wherein said apparatus is a server of the Packet-switched StreamingService PSS of a third generation mobile communications system.